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Abstract 

Student enrollment in K—12 online learning programs showed a tenfold expansion in the 
years between 2002 and 2005. Despite increased implementation to fulfill critical local needs, 
there is very little evidence-based research available to inform education leaders’ decisions 
relating to these initiatives. To address the important question of whether online learning 
can be as effective as traditional face-to-face learning, this research presents the findings from 
a quasi-experimental design implemented to examine the effect of the Louisiana Algebra I 
Online initiative on student outcomes. The findings presented suggest that the Louisiana 
Algebra I Online model is a viable online model for providing effective Algebra I instruction. 
(Keywords: online learning, algebra I, virtual learning, distance learning.) 

Increasing numbers of schools and districts are implementing online courses 
for students. Estimates of student enrollment in K—12 online learning programs 
show a tenfold expansion in four years, from an estimated 40,000-50,000 students 
during the 2001-2002 school year, to more than 520,000 in the 2004-2005 school 
year (McCleod, Hughes, Brown, Choi, & Maeda, 2005). In addition to helping 
address shortages of qualified teachers in specific subject areas and geographical 
regions, online learning also offers schools and districts the opportunity to broaden 
the variety of course offerings available to students (Perreault, Waldman, & Zhoa, 
2002). These advantages are particularly salient given the current mandates of 
No Child Left Behind for improving student achievement and for providing 
all students with highly qualified teachers. Rural schools are leading the way in 
online course enrollment with 46% of rural districts reporting that their students 
were participating in distance education courses compared with 28% in urban 
school districts and 23% in suburban districts (Setzer & Lewis, 2005). Within the 
umbrella term “distance education” or “online learning” there are myriad models 
through which coursework can be delivered. Lor example, during the 2002-2003 
school year, many districts reported that they used two-way videoconferencing 
to teach classes (55%) while others described asynchronous computer-based 
instruction delivered via the Internet (47%) (Setzer & Lewis, 2005). Other 
online models include pre-recorded video instruction, synchronous computer- 
based courses delivered via the Internet, or using other technologies such as 
teleconferencing or CD-ROM to provide instruction to students. 

While online learning programs are becoming more widely used by K—12 schools 
and districts, there is very little evidence-based research available to guide education 
leaders’ choice of initiative. Lor example, there has been very little high quality 
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research on the effectiveness of online learning at the high school level compared 
to traditional face-to-face learning, and almost no research on curriculum specific 
interventions. For example, in a meta- analysis of 19 research studies that examined 
K-12 online programs, Cavanaugh (2001) reported that there was preliminary 
evidence that suggests that students in online courses do as well as students who 
receive face-to-face instruction. However, only 1 0% of the studies examined by 
Cavanaugh employed an experimental or quasi-experimental approach. Similarly, 
another meta-analysis conducted by Ungerleider and Burns (2002) found that only 
two of the 232 studies that focused on online learning at the K-12 level met their 
criteria of being “empirically sound” (i.e., used experimental or quasi-experimental 
methods, used appropriate data analyses, and made conclusions appropriate to the 
methodology used). 

Looking to the research that has been conducted in online learning environments 
in higher education settings, several studies have found that online learning 
programs provide students and teachers with quite different learning and teaching 
environments than those provided in traditional, face-to-face settings (Bernard 
et al., 2004; Peters, 2003; Stodel, Thompson, & MacDonald, 2006). The 
distinguishing characteristics include class structure (Paloff & Pratt, 2001), the 
level and types of peer-to-peer and student-teacher interactions that take place 
(Bernard et al. 2004; Lally & Barrett, 1999), and the lack of “social presence” 
during the learning process (Aragon, 2003; Bibeau, 2001; Garrison & Cleveland- 
Innes, 2005). While many studies of higher education learning outcomes have 
found no statistical differences in achievement for online courses compared to 
traditional, face-to-face courses, there remains concern that the quality and depth 
of learning may be affected in online courses (Allen, Bourhis, Burrell, & Mabry, 
2002; Lockyer, Patterson, & Harper, 2001; Neuhauser, 2002; Thirunarayanan & 
Peres-Prado, 2001-2002). Mediating variables that may promote or hinder learning 
success in an online environment have been isolated in several studies and these 
include students’ attitudes towards the online environment (Sherry, Fulford, & 
Zhang, 1998) and their beliefs about the efficacy of online learning (Cooper, 2001; 
Allen et al., 2002; Muilenburg & Berge, 2001; Navarro & Shoemaker, 2000; Smith 
& Dillon, 1999). 

While much of the available research on online learning focuses on higher 
education settings, these studies provide us with a framework for understanding 
how and why student outcomes in K— 12 online distance learning programs may 
be different from those in traditional, face-to-face courses. The current lack of 
sound empirical evidence about the impacts of online learning in K-12 settings 
is troublesome given its widespread and growing use and the costs incurred from 
limited school budgets to support its use. Since many online learning initiatives are 
implemented to fulfill critical local needs, the important question is whether online 
learning can be as effective as traditional, face-to-face learning. That is, do students 
who participate in online learning initiatives perform as well as students who 
participate in traditional, face-to-face courses? Also of interest is whether students in 
online learning programs engage in different types of peer-to-peer interactions than 
students in traditional courses and whether they differ in how they perceive their 
learning. With these questions in mind, we conducted this research to examine how 
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student outcomes from one particular type of K-12 online distance learning model 
compared with those of students in a traditional, face-to-face course. 

The 2004-2005 school year implementation of the Louisiana Algebra I Online 
project provided a unique opportunity to use a quasi-experimental design to 
examine these questions. This online distance learning initiative, a program of 
the Louisiana Virtual School (LVS), was first implemented in 2002. During the 
2004-2005 school year, 257 students from 18 intact classrooms in six school 
districts and two private schools participated in the initiative. Designed for schools 
in which a sufficient number of certified mathematics teachers were not available, 
the goal of the initiative was to improve educational opportunities for students by 
providing them with a high-quality, standards-based curriculum delivered online by 
a certified mathematics teacher. Additionally, this highly qualified teacher mentored 
a classroom teacher who was not certified to teach mathematics. The innovation of 
the Louisiana model is that it enables a school to have a certified teacher available 
when one is not locally present, while still providing students with the structure and 
opportunities afforded by regular class meetings. 

With its large number of rural school districts in which teacher recruitment 
and retention remain problems, Louisiana faces a particular challenge in helping 
students gain proficiency in mathematics. According to Beeson and Strange (2003), 
26.3% of public school students in Louisiana are enrolled in rural school districts. 
Of this percentage, 3 1 .7% of the students are classified as minority students and 
21.5% live in poverty. Overall, 90.5% of Louisiana’s teachers were classified as 
“highly qualified” during the 2003-2004 school year. However, when the data 
are disaggregated by school poverty level, it is evident that high poverty schools in 
Louisiana have lower percentages of highly qualified teachers. For example, during 
the 2003-2004 school year, only 86.5% of the teachers were considered highly 
qualified in the high poverty schools, compared to the 93.1% of teachers in low 
poverty schools (2003-2004 Louisiana State Education Progress Report, http:// 
www.doe.state.la.us). 

THE LOUISIANA ALGEBRA I ONLINE PROJECT 

Students enrolled in the Louisiana Algebra I Online program participated in 
an algebra class that met on a standard class schedule, with all students meeting 
together in a technology-equipped classroom. Each student had a multimedia, 
Internet-connected computer during class time where the course curriculum and 
materials could be accessed. Students who had access to an Internet-connected 
computer at home or elsewhere could access their course materials outside of 
the school day. Each student benefited by having two teachers: an experienced 
secondary certified mathematics teacher who was available online and a teacher 
available in the classroom who was not certified to teach mathematics. In addition 
to giving students access to two teachers, this approach provided a unique 
professional development model for the in-class teachers since they worked 
throughout the year with a highly qualified mathematics teacher who was available 
online. 

The Algebra I Online course used a curriculum designed by the Louisiana 
Center for Educational Technology (LCET) and the Louisiana School for Math, 
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Science, and the Arts (LSMSA). The course was aligned to the National Council 
of Teachers of Mathematics (NCTM) standards and the Louisiana state content 
standards, benchmarks, and Grade Level Expectations (GLEs) (http://lvs.doe.state. 
la. us ). Delivered via the Blackboard online learning platform, the course design 
incorporated e-mail, Internet tools such as Java Applets, and video into the lessons. 
Students also used graphing calculators and Grapbire 2 Digital Tablets™ With a 
stylus and handwriting capture system. Students in the comparison classrooms 
received the traditional or “business as usual” approach to algebra instruction and 
the curriculum and textbooks used in both the treatment and comparison group 
classrooms addressed the same content standards and learning goals as outlined in 
the Louisiana GLEs (http://www.louisianaschools.net). 

The online teacher served as the teacher of record for the students in the Algebra 
I Online classrooms and as a mentor and model to the in-class teacher. The online 
teachers were expected to respond to the students’ questions and assignments in 
a timely manner, to provide feedback on homework, tests, and discussion board 
postings, to keep the students and the in-class teacher informed about student 
progress and status, maintain the schedule in the course delivery system used 
for the online course, provide whole group and individual communications and 
instruction, and stay in constant contact with the in-class teacher. The online 
teachers reported grades to LVS on a regular basis and maintained e-mail backups 
to submit to LVS three times a year. 

The in-class teachers, most of whom were either certified in other areas or 
were working toward a secondary mathematics teaching credential, followed the 
curriculum guide to facilitate face-to-face learning activities and collaborated with 
the online teachers in guiding and supporting the students. Their responsibilities 
included creating an atmosphere conducive to learning in the classroom, assisting 
students with technology, supervising and guiding the Algebra I students through 
the coursework, working with students when problems arose, staying in constant 
contact with the online teacher, providing feedback on activity days, and proctoring 
tests and exams. Both the in-class and online teachers participated in workshops to 
prepare them for participation in the Algebra I Online initiative (Kleinian, Carey, 
Halsted, & O’Dwyer, 2005). 

In the first year of the course delivery (2002-2003), the Southwest Educational 
Development Laboratory (SEDL) included the Algebra I Online course as part 
of a study to determine the quality and effectiveness of curriculum, instruction, 
and student assessment in online courses. The study reported that the Algebra I 
Online program “meets the criteria” of an effective and quality Web-based course 
for middle and high school students based on (1) alignment with state standards, 

(2) providing resources that enrich course content, (3) providing opportunities for 
students to engage in abstract thinking and critical reasoning skills, (4) providing 
appropriate student-to-student, as well as teacher-to-student interactions. 

STUDY DESIGN AND SAMPLE CHARACTERISTICS 

A quasi-experimental design was used to examine the impact of the Louisiana 
Algebra I Online initiative on student outcomes. Under this design, student 
outcomes for those receiving the treatment were compared with those of students 
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in matched comparison classrooms. Implemented in 18 intact classrooms in six 
school districts and two private schools, a total of 257 students participated in the 
Louisiana Algebra I Online project during the 2004-2005 school year. To enroll 
in the Algebra I Online program, districts were required to demonstrate a need for 
certified mathematics teachers and a desire to provide professional development 
to teachers interested in obtaining certification in secondary mathematics. 

District leaders were asked to identify the schools in which the program was to be 
implemented, the teachers who would serve as in-class teachers for the Algebra I 
program, and a comparison classroom. 

According to the NCES 2001-02 School District Locator, four participating 
districts qualified as rural or small town districts and the remaining two were 
classified as urban fringe of mid-size city districts. All six districts had previously 
participated in the Algebra I Online program during the 2003-2004 school year, 
and two had participated in the pilot year in 2002-03. The Algebra I Online 
course was available to students in Grades 8 and 9 and there were no academic 
prerequisites for enrolling in the course. The course was not intended for those who 
had previously failed algebra, so regardless of grade, students were taking Algebra 
I for the first time. Since online learning makes certain demands upon students 
beyond those in face-to-face algebra courses, the districts and schools were asked to 
avoid selecting students on the basis of prior mathematics achievement and instead 
to consider students’ ability to work independently and communication skills when 
selecting students for the Algebra I Online course. Students who were deemed 
eligible to participate in the project were then assigned to classrooms that would 
receive the intervention. 

The majority of teachers who served as in-class teachers for the Algebra I Online 
program did not hold a secondary mathematics certification. Approximately 50% 
of these teachers were certified to teach elementary education and the remainder 
held certifications in middle school mathematics, special education, health and 
physical education. The average number of years teaching for the in-class teachers 
was 8.5, and they taught an average of five classes per semester with an average class 
size of 21 students. 

Districts were asked to identify comparison classrooms that were similar with 
regard to mathematics ability, environment, and size, but where teachers used 
traditional “business as usual” approaches to teaching algebra. In Louisiana, the 
only requirement for a traditional algebra classroom is for the teachers to cover 
the Louisiana content standards and benchmarks, as well as the Grade Level 
Expectations (http://lvs.doe.state.la.us). While there was ongoing professional 
development on effective classroom strategies, there was no statewide mandate on 
how the classes should be taught. Since the comparison group teachers were not 
required to alter instruction it was anticipated that these classes might make use 
of some technologies. In fact, when survey data from the comparison classrooms 
were analyzed, 76.2% of students reported using technology during their 
traditional, face-to-face algebra course. Of these students, 83.7% said they used a 
graphing calculator, 9.8% said they used online simulations, 17.6% said they used 
spreadsheets, and 2% reported using e-mail. 
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Eighty-five percent of the teachers in the comparison classrooms were certified to 
teach secondary math. The average number of years teaching for these teachers was 
1 1.7, and similar to the in-class participating teachers, teachers in the comparison 
classrooms reported teaching an average of five classes per semester with an average 
class size of 22 students. 

The online teachers were selected on the basis of their outstanding teaching 
credentials and were identified by the Touisiana Department of Education to be 
at the level of “mentor teachers.” They were master teachers who were certified to 
teach secondary mathematics and experienced with technology. 

To assess the fidelity of the implementation of the Algebra I Online program, 
classroom observations were conducted in a convenience sample of the treatment 
group classrooms; nine of the 1 8 classrooms made up this convenience sample. 

The classroom observations documented the types of instructional phases (i.e., 
transition, whole group instruction/discussion, student individual work, etc.), the 
roles fulfilled by the in-class teacher, and types of student interactions across the 
treatment classrooms. Additional observational data recorded how the technologies 
were used and how the classrooms were set-up. With two trained observers in each 
of nine classrooms, the classroom observation data showed high levels of similarity 
among the activities being conducted across the treatment classrooms, and in each 
case these activities were directly linked with how the course content was presented. 
For example, in each of the treatment classrooms, students used technology to 
“analyze information,” “evaluate information,” and “self-assess or assess” their own 
work or work from classmates (Kleiman, Carey, Halsted, & O’Dwyer, 2005). 

INSTRUMENTS 

Three instruments were used to gather data from students in the treatment 
and comparison classrooms. These were (a) a pretest designed to assess general 
mathematics ability, (b) a posttest aligned with Louisiana’s GLEs in Algebra I, 
and (c) a survey to gather data about students’ experiences in both the traditional 
face-to-face course and the Algebra I Online course (Kleiman, Carey, Halsted, & 
O’Dwyer, 2005). 

Administered in September 2004, the pretest comprised 25 multiple-choice 
items that assessed students’ general mathematics ability and mathematics 
comprehension. Scores from the pretest were used to examine the equivalence of 
the treatment and comparison groups prior to the implementation of the Algebra I 
Online Project. The Cronbach’s alpha estimate of reliability for the pretest was 0.70. 
The posttest, administered in June 2005, was also a 25-item multiple choice test 
but was aligned with the Algebra I Online course objectives and Louisiana’s GLEs 
outlined by the Louisiana Department of Education. The Cronbach’s alpha estimate 
of reliability for the posttest was 0.81. 

To ensure the content validity of the posttest for assessing Algebra I learning, two 
mathematics educators with experience developing curriculum and assessments 
mapped the items on the test to the Louisiana GLE and Algebra standards, rated 
them for levels of difficulty, and subsequently selected 20 to represent a balanced 
sample of the Louisiana GLEs with varying levels of difficulty (Kleiman, Carey, 
Halsted, & O’Dwyer, 2005). 
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To examine criterion validity evidence for these assessments, the correlations 
between the pretest and posttest scores and students’ standardized test scores 
(collected in spring 2005) were calculated. At grade eight, the Louisiana 
Educational Assessment Program (LEAP) assessment scores for students in the 
Algebra I Online course were compiled. Similarly at grade nine, student scores on 
the norm-referenced Iowa Test of Educational Development (Iowa) assessment 
(http://www.doe.state.la.us) were compiled. For both tests, only an overall math 
score was available and so it was not possible to separate the Algebra strand from the 
general mathematics score. Despite this, the correlations between the eighth grade 
LFAP data and the pretest and posttest scores were moderately strong and positive 
at 0.55 and 0.67, respectively. For the ninth grade Iowa data, the correlations were 
0.50 and 0.55, respectively. In terms of criterion validity evidence, the magnitude 
and direction of the relationship suggests that scores on the pre- and posttest 
measures were related in a valid way to scores on the state tests. 

Surveys were administered to all students to gather additional information 
about students’ experiences and their reactions to their Algebra I courses. Due to 
student confidentiality concerns, students’ survey responses could not be linked 
to individual students and their test scores. For this reason, student survey data 
provided only general information about student characteristics and their views 
about their participation in the project. A 38-item survey was administered online 
to students in the treatment classrooms. This instrument was designed to collect 
data about students’ confidence levels with algebra and technology, their experience 
in class, their use of technology, and how they interacted with one another and their 
teachers. 

Students in the comparison classrooms were administered a 10 item paper- 
and-pencil survey that was designed to collect data about the comparison group 
students’ comfort level with algebra and technology, their experience in the class, 
their use of technology during their traditional, face-to-face course, and how they 
interacted with one another in the course. 

For the treatment group, pretest achievement data were gathered from 26 1 
students and posttest data were collected from 23 1 students. Of the 23 1 students 
who completed both the pretest and posttest, survey responses were collected from 
213; 179 eighth grade students and 52 ninth grade students. This resulted in a 
survey response rate of approximately 92% in the treatment group. Within the 
comparison classrooms, pretest data were collected from 360 students and posttest 
data from 232 students. Of the 232 students that completed both the pretest 
and posttest, survey responses were gathered from 207 students; 9 1 eighth grade 
students and 14 1 ninth grade students. The resulting survey response rate in the 
comparison group was approximately 89%. Since students in the treatment group 
and comparison groups were taking Algebra I for the first time regardless of grade, 
data from both grades were combined for analysis. 

RESULTS 

In the treatment and comparison classrooms, pretest achievement data were 
gathered from 621 students and posttest data were collected 463 students. Since 
158 students who had taken the pretest did not take the posttest, patterns of non- 
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Table 1: Pretest and posttest means for treatment and comparison groups 




N 

Mean 

Standard 

Deviation 


Comparison group 

232 

14.99 

3.33 

Total Pretest Score 


Treatment group 

231 

14.91 

2.88 


Comparison group 

232 

14.61 

5.37 

Total Posttest Score 


Treatment group 

231 

15.27 

4.50 


completion were examined. Approximately half of the students who were missing 
posttest scores appeared to be missing at random within classrooms. The remaining 
students with missing posttest scores appeared to be from comparison group 
classrooms where the range of missing scores within classrooms was between 50% 
and 100%. The mean pretest score for all students with missing posttest scores 
(13.25) was statistically significantly lower than the mean for the students who 
had taken both the pretest and the posttest (14.90) (t = -3.99, df = 146, p < .001). 
The analyses of student outcomes presented here were conducted using only those 
students for whom pre- and post-invention outcomes were available for a total of 
463 students. 

Pretest data were used to examine the equivalence of the treatment and 
comparison group classrooms prior to the implementation of the intervention. 
With a maximum possible score of 25 points, Table 1 shows that the mean pretest 
score for the comparison group students was 14.99 points, and the mean for the 
treatment group was 14.91 points. The difference between the treatment and 
comparison group means on the pretest was not statistically significant (t = 0.27, df 
= 461, p = 0.788). 

The mean posttest score for the comparison group was 14.61 points and the 
treatment group mean was 15.27 points. Similar to the pretest, the comparison 
group posttest scores were slighdy more variable than the treatment groups’ scores; 
5.37 compared to 4.50. 

Pre-to-posttest gain scores were not examined because the pre- and posttests were 
not psychometrically equivalent. Instead, a multilevel regression model in which 
students pretest scores were included as a covariate was used to examine the effect 
of the intervention on students’ posttest scores. Specifically, students were modeled 
as nested within classrooms, and posttest scores were modeled as a function of 
students’ pretest scores at level- 1 and a dummy variable representing treatment 
or comparison group membership at level-2. Since the intervention was applied 
at the classroom level and students were nested within classrooms, this approach 
accounted for the hierarchical nature of the data (Raudenbush & Bryk, 2002). 

In total, 463 treatment and comparison group students were nested within 33 
classrooms. 

Prior to inclusion in the two-level model, students’ pretest scores were adjusted 
for measurement error or unreliability (recall that the Cronbach’s alpha for the 
pretest was 0.70). While measurement error in the pretest and/or posttest may 
reduce the precision of the estimate of the treatment effect; measurement error in 
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Table 2: Reliability- Adjusted Two-Level Model to Predict Posttest Scores 



Coefficient 

St. error 

p-value 

Level- 1 Model 




Adjusted Pretest Scores 

0.74 

0.08 

.000 

Level-2 Model 




Group Membership 

1.85 

1.07 

.093 


the pretest in particular may also introduce a bias when estimating the regression 
slope and in turn produce a biased estimate of the treatment effect (Shadish, Cook, 
& Campbell, 2002). By correcting for unreliability in the pretest measure, the 
regression coefficient in the multilevel model for the pretest scores was likely to be 
an unbiased estimate of the population regression coefficient. 1 

The results in Table 2 show the coefficients and the related significance levels 
for the two-level regression model that included students’ adjusted pretest scores 
at level- 1 and students’ membership in a treatment or comparison classroom at 
level-2. The model explained approximately 30% of the variability in students’ 
posttest scores. At level-1, the adjusted pretest scores were significant predictors of 
students’ posttest scores (p < 0.001). The level-2 coefficient for the dummy variable 
representing membership in the treatment or comparison group (1.85) indicates a 
positive effect for group membership; that is, membership in the treatment group 
was associated with slightly higher posttest scores. However the coefficient was not 
statistically significant (p = 0.093) indicating that after controlling for scores on 
the pretest, group membership was not statistically significantly related to students’ 
posttest scores. These results suggest that students in the participating Algebra I 
Online classrooms appear to have performed as well on the posttest as students 
who participated in the traditional algebra I course. This finding is similar to that 
reported in other studies which compared learning outcomes for online and face-to- 
face courses (e.g., Allen et al, 2002; Navarro & Shoemaker, 2000). 

To examine whether the groups differed in terms of their responses to particular 
types of items on the post-intervention assessment, item-level means were 
compared. The treatment classrooms’ mean was higher than the comparison 
classrooms’ mean on 18 of the 25 items included on the posttest. The largest 
group of items (5) that assessed a single skill related to students’ ability to “write an 
algebraic expression from a real-world example” and for each of these five items, the 
treatment group mean was higher than the comparison group mean. The difference 
between the groups was statistically significant for four of the five items. Of the 
seven items on which the comparison group classrooms outscored the treatment 
group classrooms, the difference between the comparison and treatment group 
means was statistically significant for only three. 


1 The adjusted pretest scores were calculated using the formula 
X adj = X + r(X - X) where r is the reliability of the pretest measure, in this case, 
Cronbach’s alpha. The corrections for treatment and comparison groups were made 
separately. 
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Table 3: Enjoyable Aspects of the Algebra I Online Course 



Percentage of students in treatment group 
agreeing that they liked this aspect of the 


course 

Using technology to learn math 

71.8% 

Working with other students 

68.5% 

New experience 

59.2% 

Activity days 

44.6% 

Challenging material 

32.9% 

Having two teachers 

30.5% 

The pace of the course 

23.9% 

Table 4: Treatment Group Students Reported Difficult Aspects of the 
Online Algebra I Course 

Percentage of students in treatment 
group agreeing that this was the 
hardest thing about the course 

Getting the assignments done on time 

70.0% 

Getting used to a new class structure 

49.8% 

Working independently 

36.6% 

Using the technology 

15.0% 


Survey responses from students in the treatment and comparison group 
classrooms were also compared to examine whether students in the online learning 
program engaged in different types of peer-to-peer interactions than students in 
the traditional, face-to-face course and whether they differed in how they perceived 
their learning. Students in the treatment group were asked several questions related 
to their experiences in the Louisiana Algebra I Online project. These questions 
related to their perception about the difficulty of the course, their experiences with 
the technology and the course materials, and the type and level of communication 
they engaged in with the in-class and online teachers. Since students in the 
treatment group were expected to use technology extensively throughout the 
course, students were asked to describe their experience level with technology before 
enrolling in the Algebra I Online course. The majority of treatment group students 
reported that they were either somewhat experienced (64.30%) or very experienced 
(26.80%) with technology before enrolling in this course. 

When asked to identify the characteristics that they most liked about the online 
course, Table 3 shows that the largest percentage of students in the treatment group 
classrooms agreed that they liked using technology to learn math (71.8%), working 
with other students (68.5%), and having a new experience (59.2%). The pace of 
the course was the least selected characteristic (23.9%). 

The students in the treatment group classrooms were asked to identify the most 
difficult aspect of the Algebra I course. Table 4 shows that 70% of students said 
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Table 5: Course Materials and Technologies Most Helpful for the Treatment 

Group Students 



Extremely 

helpful 

Very 

helpfuf 

Helpful 

Not very 
helpful 

Discovering Algebra Textbook 

9.4% 

29.6% 

49.8% 

11.3% 

Blackboard course 

20.7% 

28.2% 

39.0% 

12.2% 

E-mail 

22.1% 

32.4% 

33.3% 

12.2% 

Animated tutorials 

26.8% 

31.9% 

26.3% 

15.0% 

TI-83 Graphing Calculators 

63.4% 

19.2% 

14.1% 

3.3% 

Graphire 2 Digital Tablet ™ 

38.0% 

30.5% 

19.7% 

11.7% 


Table 6: Frequency of Students Asking In-Class Teachers about Content 

or Technology 



Every 

lesson 

Couple 
of times 
a week 

Once a 
week 

Every 
couple 
of weeks 

Never 

How often did you ask 
your in-class teacher 
questions about Algebra 
content? 

33.30% 

26.30% 

17.40% 

18.30% 

4.70% 

How often did you ask 
your in-class teacher 
questions about 
technology? 

6.60% 

12.20% 

16.90% 

37.60% 

28.80% 

Frequency of the 
communication with 
the online teacher? 

44.60% 

27.70% 

12.20% 

6.60% 

8.90% 


they found that getting their assignments done on time was the most difficult. 
While about half the students reported finding it difficult to get used to the 
new class structure, only 15% reported that using the technology was the most 
challenging aspect. The online format likely required more independent work than 
a traditional face-to-face course and only about one-third of the students reported 
having difficulty working independently. 

The treatment group was also administered survey items to gather information 
about their experiences with the course materials and technologies that were most 
helpful to them in learning algebra. Table 5 shows that the majority of treatment 
group students identified the TI-83 graphing calculator as being the most helpful; 
82.6% of treatment group students said that the graphing calculator was either very 
helpful or extremely helpful. The Graphire 2 Digital Tablet, the animated tutorials, 
and e-mail were also identified as being very or extremely helpful by the students in 
the treatment group classrooms. 

To investigate the role of the in-class teacher, the students in the treatment group 
classrooms were asked how often they approach their in-class teacher with content 
or technology questions. Table 6 shows that more than half (59.6%) the students 
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Table 7: Student Responses to the Question “Did the Course Provide a 
Good Learning Experience?” 



Yes 

Satisfactory 

No 

Comparison Group Students 

62.80% 

30.90% 

6.30% 

Treatment Group Students 

38.50% 

40.80% 

20.70% 


in the treatment group classrooms reported asking their in-class teacher about the 
algebra content either a couple of times a week or every lesson. Conversely, a smaller 
percentage of students reported asking their in-class teachers about technology more 
than once a week; more than half the students in the treatment group classrooms 
(64.4%) reported asking their in-class teacher questions about technology either 
“never” or “every couple of weeks.” Table 6 also shows that almost half (44.6%) of 
the treatment group students said that they communicated with their online teacher 
during every lesson. Fewer than 9% of treatment group students reported that they 
almost never had e-mail communication with their online teacher. 

On a similar note, the students in the treatment group classrooms were asked to 
share their perceptions about the adequacy of the interaction between them and 
the online teacher. When asked whether the online teacher-student interaction was 
adequate for their needs, only 38% of the treatment group students said that the 
interaction was adequate. When asked how helpful their online teacher was for 
understanding algebra, approximately three-quarters of treatment group students 
said that the online teacher was either “extremely helpful,” “very helpful,” or 
“helpful.” 

Several common survey items were administered to the treatment and 
comparison group students to examine their Algebra I learning experiences. Table 
7 shows that the majority of students in both groups reported that they had a good 
or satisfactory learning experience in their respective Algebra I courses. A higher 
percentage of treatment group students reported that they did not have a good 
learning experience; 20.70% in the treatment group compared to 6.30% in the 
comparison group. While these findings are similar to those of Allen, Bourhis, 
Burrell, and Mabry (2002) who found that not all students were satisfied with their 
online learning experiences or place a similar value on online learning, other studies 
have found that students in online and traditional settings have similar levels of 
satisfaction with their learning experience (Cooper, 2001; Smith & Dillon, 1999). 

As Table 8 shows, a higher percentage of students in the comparison classrooms 
reported feeling either confident or very confident in their algebra skills after 
taking this course (67.6%) compared to the students from the treatment group 
classrooms (49.8%). While this finding is interesting given that students in the 
treatment classrooms had a mean posttest score that was similar to the mean in the 
comparison classrooms, it is consistent with the findings from other studies that 
suggest that students in online learning courses may have poorer perceptions of 
their learning (Bernard et al., 2004). The data also show that the students in the 
treatment group classrooms reported higher rates of feeling either confident or very 
confident in their technology skills after taking this course (79.8%) than did the 
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Table 8: Student Confidence in their Algebra and Technology Skills 


Very Confident 

Confident 

Somewhat a A 

~ r t confident 

Confident .. 

at all 

Student Treatment 11.30% 

38.50% 

39.40% 10.80% 

confidence in 

algebra skills Comparison 20.30% 

47.30% 

28.50% 3.90% 

Student Treatment 41.80% 

38.00% 

17.80% 2.30% 

confidence in 

technology skills Comparison 13.60% 

48.50% 

34.00% 3.90% 

Table 9: Student Interaction in Treatment and Comparison Classrooms 


Treatment Group Comparison 


Students 

Group Students 

To talk about the math in the course 

84.0% 

68.0% 

To talk about the technology 

55.9% 

Not asked 

To socialize 

62.4% 

64.1% 

To understand assignment directions 

71.4% 

67.7% 

To work together on in-class assignments 

70.9% 

70.1% 

To work together on homework 

55.9% 

51.5% 

To work together on activities 

86.9% 

60.9% 


students from the comparison group (62.1%). This is not surprising considering the 
online nature of the course. 

Examination of peer-to-peer interactions was deemed important because the lack 
of “social presence” during the learning process has been found to be an important 
mediating variable for learning and attitudinal outcomes (Aragon, 2003; Bibeau, 
2001; Garrison & Cleveland-Innes, 2005). For this reason, students in both 
groups were asked about the circumstances under which they interacted with other 
students in their Algebra I classroom. As the results in Table 9 show, the students 
in the comparison classrooms reported spending less time interacting with other 
students to talk about the math in the course; 68% in the comparison classrooms 
compared to 84% in the treatment classrooms. The comparison group students also 
spent less time working together on activities than the students in treatment group 
classrooms; 60.9% compared to 86.9%. The amount of time spent socializing, 
interacting to understand assignment directions, and working together on both 
in-class assignments and homework were approximately equivalent across the two 
groups. It would appear from the data that there was no perceived lack of “social 
presence” among the treatment group students. This may have been due to the 
hybrid nature of the course, whereby students benefited from meeting on a regular 
class schedule with an in-class teacher and their peers while also having access to a 
highly-qualified teacher online. 
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DISCUSSION AND CONCLUSIONS 

The Louisiana Algebra I Online model was designed and implemented to bring 
highly qualified mathematics teachers to students in places where they would not be 
otherwise available, to provide students with the structure of a regular class period, 
and to provide a unique professional development model for local teachers. To 
address the critical need for evidence-based research to inform the proliferation of 
online distance learning initiatives, this paper described a quasi-experiment that was 
conducted to examine the effect of one particular model for online learning at the 
high-school level on student performance; namely, the Louisiana Algebra I Online 
model. To complement the learning outcomes, survey responses from students in 
the treatment and comparison group classrooms were examined. Recognizing that 
content delivery methods (online versus traditional, face-to-face delivery) should 
not hinder student learning, it was important for us to understand the experiences 
of students in the treatment and comparison classrooms during their participation 
in the Algebra I Online course (Bernard et al., 2004; Paloff & Pratt, 2001; Peters, 
2003; Stodel, Thompson, & MacDonald, 2006). 

The survey data showed that the students in the treatment classrooms appeared 
to have had adequate access to technology, enjoyed using technology to learn math, 
and enjoyed the new learning experience. The students identified the characteristics 
of the Algebra I Online course that were most helpful to them in learning algebra 
and these included graphing calculators, Graphire 2 Digital Tablet hardware, as 
well as the animated tutorials and e-mail communications. The data showed that 
in terms of implementation, the project was successful in providing adequate 
and reliable technology access to those students participating in the course. Also, 
with the majority of students reporting that they liked to use technology to learn 
mathematics, students in the treatment classrooms appear to have responded 
positively to the online learning environment in which the mathematics content 
was presented. 

The Algebra I Online model also required changes to the type and level of 
interaction between students and teachers, and among the students in the class. The 
majority of students in the treatment classrooms felt that the interaction with their 
online teacher was somewhat less than adequate or should have been a lot more. 
With regard to peer-to-peer interactions, the students in the treatment classrooms 
reported spending more time interacting with other students to talk about the 
content of the course and working together on course activities. The amount 
of time spent socializing, interacting to understand assignment directions, and 
working together on both in-class assignments and homework were approximately 
equivalent across the two groups. The types of interactions among the students 
in the treatment classrooms appear to contradict the commonly held concern 
that students in K-12 online classes will not remain on task. As studies in higher 
education have shown, online programs can often isolate students during the 
learning process (e.g., Bernard et al. 2004; Lally & Barrett, 1999). While the 
Louisiana Online Algebra I program is unusual in that it provides students with 
the benefit of a regular class schedule where they interact with their peers as well 
as an online teacher, other emerging K— 12 online models may not provide such 
opportunities for important socio-academic interactions. As more K— 12 online 
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distance learning models emerge, the ways in which peer-to-peer interactions 
manifest will no doubt continue to evolve and studies of these interactions should 
be undertaken. This is particularly important given the important role that social 
interaction plays during the K-12 years. 

When compared to the comparison group students, a higher percentage of 
students in the treatment group classrooms reported that they did not have a good 
learning experience. Considering the same content standards were covered in both 
groups, the difference in learning experience may have been a function of the 
newness of the online model, specific differences in the curricula, and/or differences 
in classroom approaches. Although some studies in higher education settings have 
found there to be no difference between students’ level of satisfaction in online and 
traditional courses (Cooper, 2001; Smith & Dillon, 1999), other studies point to 
fundamental differences between the learning experiences that lead some students 
to be less satisfied with their online experience (Allen, Bourhis, Burrell, & Mabry, 
2002). Similarly, students in the treatment and control classrooms differed with 
respect to their confidence in their algebra skills. Despite having similar posttest 
means, compared to the comparison group, a lower percentage of students in the 
treatment classrooms reported feeling either confident or very confident in their 
algebra skills after the course. This finding supports those from studies described by 
Bernard et al (2004) in their meta-analysis on how online and traditional classroom 
experiences vary. It may be that the model of delayed feedback and dispersed 
authority in the online course led to a “lost” feeling where students could gauge 
“how they were doing.” There is currendy a lack of research on the mediating 
or moderating effect of student satisfaction and confidence levels on learning 
outcomes in K-12 settings. Given the importance of student attitudes and beliefs 
for moderating learning outcomes in traditional, face-to-face learning environments 
(Koretz, McCaffrey, & Sullivan, 2001; O’Dwyer, 2005), future research into the 
effectiveness of online or distance learning initiatives in K-12 settings should pay 
particular attention to student satisfaction and other affective measures. 

Despite their feeling less confident in their algebra skills, the data showed that 
students in the treatment classrooms outscored the comparison classrooms on 1 8 
of the 25 items on the posttest. The data also showed that students in the treatment 
classrooms tended to do better than students in the comparison classrooms on the 
group of items that required them to create an algebraic expression from a real- 
world example. Bearing in mind that the content standards covered in both courses 
was the same, the differential effect of the online distance learning initiative on 
particular types of algebra skills should be explored further. It may be that students 
in the treatment classrooms acquired better conceptual understanding of some 
aspects of the content due to the nature of the technology-enhanced teaching 
tools employed. Further research would need to be conducted to substantiate 
this hypothesis. Overall, students in the treatment group performed as well as 
students in the comparison face-to-face classrooms. While this finding is similar 
to that reported in other studies that compared learning outcomes (Allen et al, 
2002; Navarro & Shoemaker, 2000), more research is needed at the K-12 level to 
examine this important issue. With more than half a million elementary, middle 
and high school students impacted by some form of online learning initiative 
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during the 2004-2005 school year, it is vital that the educational community and in 
particular state and local decision-makers have access to high-quality research they 
can use to inform their ongoing investments in online learning initiatives. 

Although there are many types of online and distance learning models available 
for schools and districts, the results of this research suggest that the Louisiana 
Algebra I Online model is a viable approach to providing Algebra I instruction, 
particularly when a certified mathematics teacher is not available locally. With the 
recent increase in investments for online distance learning initiatives at the K— 12 
level, there will be a continuing need for sound empirical evidence about the effect 
of these programs on teaching and learning outcomes, and in particular on student 
performance. 
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